Distributed Threshold Querying of General Functions by a Difference of Monotonic Representation

The goal of a threshold query is to detect all objects whose scoreexceeds a given threshold. This type of query is used in many set-tings, such as data mining, event triggering, and top-k selection.Often, threshold queries are performed over distributed data. Givendatabase relations that are distributed over many nodes, an object’sscore is computed by aggregating the value of each attribute, apply-ing a given scoring function over the aggregation, and thresholdingthe function’s value. However, joining all the distributed relationsto a central database might incur prohibitive overheads in band-width, CPU, and storage accesses. Efficient algorithms required toreduce these costs exist only for monotonic aggregation thresholdqueries and certain specific scoring functions.We present a novel approach for efficiently performing generaldistributed threshold queries. To the best of our knowledge, this isthe first solution to the problem of performing such queries withgeneral scoring functions. We first present a solution for mono-tonic functions, and then introduce a technique to solve for otherfunctions by representing them as a difference of monotonic func-tions. Experiments with real-world data demonstrate the method’seffectiveness in achieving low communication and access costs.